fix(amazonq): Inline test after function addition, writeToTextEditor fix, and package.json pathing fix #7800

surajrdy-aws · 2025-08-01T19:34:58Z

Problem

The inline test cannot be run consecutively with other tests in our suite since it switches its focus to the editorView and textEditorView. Each time the inline test is run, a new driver was launched with caused a new VSCode instance to instantiate.

The package.json should only target the .test.js files. Right now it targets all .js files in the directory.

The writeToTextEditor function is faulty since it is wrongly indexed due to its element not being accessed before counting and referencing its indices within code.

Solution

Implemented an after function which correctly switches back to our webviewView (AmazonQ). Added the true inline test into the suite and fixed driver problem.

Changed the path referenced in the package.json to .test.js files.

Fixed the writeToTextEditor function such that it includes a "dummy" space input into the textEditor and then counts the number of indices of lines. This then writes the desired text within the correct line without problems.

Treat all work as PUBLIC. Private feature/x branches will not be squash-merged at release time.
Your code changes must meet the guidelines in CONTRIBUTING.md.
License: I confirm that my contribution is made under the terms of the Apache 2.0 license.

github-actions · 2025-08-01T19:35:11Z

This pull request implements a feat or fix, so it must include a changelog entry (unless the fix is for an unreleased feature). Review the changelog guidelines.
- Note: beta or "experiment" features that have active users should announce fixes in the changelog.
- If this is not a feature or fix, use an appropriate type from the title guidelines. For example, telemetry-only changes should use the telemetry type.

laura-codess · 2025-08-01T20:47:58Z

packages/amazonq/test/e2e_new/amazonq/tests/inline.test.ts

+        // Switch back to Webview Iframe when dealing with external webviews from Amazon Q.
+        await editorView.closeAllEditors()
+        await webviewView.switchToFrame()
+        testContext.webviewView = webviewView


Is this line for setting the testContext to webview needed? In the sense that the webview isn't being update or changed so should the state be updated?

Oh smart, yeah that isn't needed

laura-codess · 2025-08-01T20:48:51Z

packages/amazonq/test/e2e_new/amazonq/tests/inline.test.ts

        const input = new InputBox()
-        await input.sendKeys('Write a simple sentece')
+        await input.sendKeys('Generate the fibonacci sequence through iteration')
        await input.sendKeys(Key.ENTER)


Does this enter now accept the generation?

Unfortunately it does not, but it is required for us to get out of the suggestion decision messing up any following clean ups

laura-codess · 2025-08-01T20:50:22Z

packages/amazonq/test/e2e_new/amazonq/utils/generalUtils.ts

 * @returns Promise<void>
 */
 export async function writeToTextEditor(textEditor: TextEditor, text: string): Promise<void> {
+    await textEditor.typeTextAt(1, 1, ' ')


I know the reason why we need a space, but should we add a comment since its not obvious?

Hweinstock

Nice work! So with this PR can we run all the tests together?

Hweinstock · 2025-08-04T12:57:33Z

packages/amazonq/test/e2e_new/amazonq/tests/inline.test.ts

-        const driver = textEditor.getDriver()
-        await pressKey(driver, 'ENTER')
-        await pressKey(driver, 'TAB')
+        await sleep(8000)


what is this sleep for? 8 seconds sounds like a lot

Good catch, the sleep is for waiting for the AmazonQ response to be generated. I realized its never always the same and can take sometime to either reason or just start coding right away. 8 seconds seems like the sweet spot for leeway for waiting for a response to be generated fully on other devices as well.

I see. Is there a way we can wait dynamically with a cap? Like basically poll every 1 second until the response shows up the in the DOM. I think this general util will be helpful for other areas as well.

Good point, will look into it!

Hweinstock · 2025-08-04T12:59:20Z

packages/amazonq/test/e2e_new/amazonq/tests/inline.test.ts

+
+        const textAfter = await textEditor.getText()
+        assert(textAfter.length > textBefore.length, 'Amazon Q should have generated code')
+        assert(textAfter.includes('fibonacci'), 'Generated code should contain fibonacci logic')


Could this be potentially flaky? Wondering if the model ever generates a solution where they use fib or hallucinates something.

Good point, fixed!

Hweinstock

I think we should avoid the sleep, and create some type of waiting utility, but otherwise this LGTM.

surajrdy-aws · 2025-08-04T19:00:43Z

/retryBuilds

…fix, and package.json pathing fix (aws#7800) ## Problem The inline test cannot be run consecutively with other tests in our suite since it switches its focus to the editorView and textEditorView. Each time the inline test is run, a new driver was launched with caused a new VSCode instance to instantiate. The `package.json` should only target the `.test.js` files. Right now it targets all .js files in the directory. The `writeToTextEditor` function is faulty since it is wrongly indexed due to its element not being accessed before counting and referencing its indices within code. ## Solution Implemented an `after` function which correctly switches back to our webviewView (AmazonQ). Added the true inline test into the suite and fixed driver problem. Changed the path referenced in the `package.json` to `.test.js` files. Fixed the `writeToTextEditor` function such that it includes a "dummy" space input into the textEditor and then counts the number of indices of lines. This then writes the desired text within the correct line without problems. --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.

fix: add inline test

0d506f1

surajrdy-aws requested a review from a team as a code owner August 1, 2025 19:34

surajrdy-aws changed the title ~~fix(amazonq): Inline test after function addition~~ fix(amazonq): Inline test after function addition, writeToTextEditor fix, and package.json pathing fix Aug 1, 2025

laura-codess reviewed Aug 1, 2025

View reviewed changes

fix: PR comments

79b1b3a

Hweinstock reviewed Aug 4, 2025

View reviewed changes

fix: PR comments from Harrison

92c3f61

Hweinstock approved these changes Aug 4, 2025

View reviewed changes

surajrdy-aws enabled auto-merge (squash) August 4, 2025 18:22

surajrdy-aws disabled auto-merge August 4, 2025 18:59

surajrdy-aws merged commit 4080c31 into aws:feature/ui-e2e-tests Aug 4, 2025
49 of 53 checks passed

fix(amazonq): Inline test after function addition, writeToTextEditor fix, and package.json pathing fix #7800

fix(amazonq): Inline test after function addition, writeToTextEditor fix, and package.json pathing fix #7800

Uh oh!

Conversation

surajrdy-aws commented Aug 1, 2025

Problem

Solution

Uh oh!

github-actions bot commented Aug 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hweinstock left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hweinstock left a comment

Choose a reason for hiding this comment

Uh oh!

surajrdy-aws commented Aug 4, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants